Information Theoretic Approaches for Predictive Models: Results and Analysis

Learning the internal representation of partially observable environments has proven to be a di cult problem. State representations which rely on prior models, such as partially observable Markov decision processes (POMDPs) are computation expensive and sensitive to the accuracy of the underlying model dynamics. Recent work by Still and Bialek o ers an information theoretic approach that compresses the available history into an internal representation and maximizes its predictive power. We propose an alternative algorithm that ensures a more accurate internal representation, and faster optimal policy convergence times. In addition, in order to validate the asymptotic nature of the theoretical algorithm, we present the rst empirical results in the eld, comparing the internal representations returned by both approaches, their accuracy, and their predictive powers.